Grading cosmetic appearance of a test object

ABSTRACT

A method includes receiving, by a processing device, a plurality of images of a test object, the plurality of images including a plurality of surfaces of the test object. The processing device selects a region of interest in each of the plurality of images, the region of interest comprising the test object having a background removed. For the plurality of regions of interest as selected, the method includes comparing, by the processing device, each region of interest with a corresponding profile image and identifying defects in each region of interest. The method includes grading, by the processing device, a cosmetic appearance of each region of interest based on the identified defects. The grades of the cosmetic appearance for each region of interest are stored.

FIELD OF THE TECHNOLOGY

At least some embodiments disclosed herein relate generally to cosmeticevaluation of an electronic device. More particularly, the embodimentsrelate to systems, devices, and methods for computer-aided cosmeticevaluation and categorization of a device such as a computing device.

BACKGROUND

Large volumes of computing devices (e.g., mobile devices, such ascellular telephones, tablets, etc.) are recycled and often refurbished.There are numerous aspects to the refurbishing process. One aspectincludes inspecting the visual characteristics of the computing deviceto grade its visual appearance. Some of these devices are thenrefurbished and can be resold to new users.

The refurbishing process requires multiple steps on different,specialized workstations, and such a multi-step process requires lots ofmanual interaction, which is both error-prone and expensive.

BRIEF DESCRIPTION OF THE DRAWINGS

References are made to the accompanying drawings that form a part ofthis disclosure and illustrate embodiments in which the systems andmethods described in this Specification can be practiced.

FIG. 1 shows a cosmetic inspection system, according to an embodiment.

FIG. 2 shows a schematic architecture for the cosmetic inspection systemof FIG. 1, according to an embodiment.

FIG. 3 shows a schematic view of a camera system of the cosmeticinspection system of FIG. 1, according to an embodiment.

FIG. 4 shows a perspective view of the camera system of the cosmeticinspection system of FIG. 1, according to an embodiment.

FIGS. 5-7 show schematic views of the camera system of FIG. 3 configuredto capture different views of a computing device, according to anembodiment.

FIG. 8 shows a flowchart for a method for inspecting a cosmeticappearance of a test object, according to an embodiment.

FIG. 9 shows a flowchart for a method for grading the cosmeticappearance of a test object, according to an embodiment.

FIG. 10 shows a flowchart for a method 300 for grading the cosmeticappearance of a test object, according to an embodiment.

Like reference numbers represent like parts throughout.

DETAILED DESCRIPTION

Computing devices such as, but not limited to, smartphones, tablets,laptops, smartwatches, and the like, can be refurbished and resold asrefurbished devices. The refurbishing process can include a cosmeticinspection to ensure that the secondhand computing device is a goodcandidate for refurbishment (e.g., in good condition such as free fromscratches, dents, cracks, and the like).

The embodiments disclosed herein are directed to a system and method forinspecting computing devices (e.g., secondhand computing devices) andgrading their cosmetic appearance.

Various examples are described herein which are directed to cosmeticinspection of a secondhand computing device. It is to be appreciatedthat the systems and methods can be applied to a computing device duringthe manufacturing process in accordance with the disclosure herein. Inan embodiment, applying the cosmetic inspection during the manufacturingprocess could be leveraged to, for example, verify quality of themanufactured product.

A system for inspecting objects (e.g., computing devices such assmartphones, tablets, etc.) and grading the cosmetic appearance of theobjects. The system includes a movable platform for receiving an objectto be inspected (“test object”). The movable platform is capable ofpositioning the test object within a semi-transparent dome. Thesemi-transparent dome is white in color, has a light transmission ratefrom at or about 0.6% to at or about 1.6%, and is shaped to match acurvature of the test objects. The dome can be made of plastics such aspolyvinylchloride (PVC). Sides of the dome may have an approximatelyhalf-cylindrical geometry. The dome has two open sides (one for entry ofthe test device and the other to provide a field of view for a camera).There are additional openings on two sides to provide a field of viewfor two additional cameras. When the images are to be captured, the testobject is located in a center of the dome (with respect to horizontaldimensions of the dome) to provide optimal lighting conditions. Lighttransmission through the dome is such that glares/reflections from thelighting on the test object are reduced, enabling high quality images tobe captured. The movable platform is capable of rotating the test objectabout a plurality of axes of rotation to capture images of all surfacesof the test object.

In an embodiment, a cosmetic inspection process can last less than at orabout 20 seconds from the time of initiation to the time the computingdevice being inspected can be withdrawn from the cosmetic inspectionsystem. In an embodiment, the cosmetic inspection process can last at orabout 16 seconds. In an embodiment, a cosmetic grade can be generated bythe system in less than at or about 30 seconds. In an embodiment, thecosmetic grade can be generated from at or about 25 seconds to at orabout 30 seconds.

The usage of cameras in the cosmetic inspection system described hereincan provide a faster output compared to prior devices which generallyrely upon scanners instead of cameras.

In an embodiment, the cosmetic inspection system can includepneumatically controlled lift and rotation systems to move the computingdevice to an appropriate location for capturing the images via thecamera system.

Captured images of the secondhand device can be compared against profileimages corresponding to a similar device, the profile images beingcaptured as images of a model assembly (e.g., good cosmetic condition).Based on the comparison, an appearance score can be generated and outputto a display of the cosmetic inspection system for display to a user.The appearance score can be a combination of each captured view of thecomputing device such that the score is impacted if any of the surfaces(sides, front, back) include cosmetic defects such as scratches, cracks,dents, or the like.

A system for cosmetic inspection of a test object is disclosed thatincludes a movable platform for receiving a test object. The movableplatform is capable of positioning the test object within a dome. Aplurality of cameras arranged oriented to capture different views of aplurality of surfaces of the test object. A plurality lights arrangedare outside the dome, the plurality of lights selectively enabled ordisabled according to which of the plurality of surfaces of the testobject is to be captured.

A system includes a housing, including a translucent dome and a movableplatform for receiving a computing device, wherein the movable platformis capable of positioning the computing device within the dome. Aplurality of cameras are arranged oriented to capture different views ofa plurality of surfaces of the computing device when the computingdevice is disposed within the dome. A plurality lights are arrangedoutside the dome, the plurality of lights selectively enabled ordisabled according to which of the plurality of surfaces of thecomputing device is to be captured.

A method includes receiving, by a processing device, a plurality ofimages of a test object, the plurality of images including a pluralityof surfaces of the test object. The processing device selects a regionof interest in each of the plurality of images, the region of interestcomprising the test object having a background removed. For theplurality of regions of interest as selected, the method includescomparing, by the processing device, each region of interest with acorresponding profile image and identifying defects in each region ofinterest. The method includes grading, by the processing device, acosmetic appearance of each region of interest based on the identifieddefects. The grades of the cosmetic appearance for each region ofinterest are stored.

A method includes capturing an image of a first surface of a computingdevice with a first camera. The method further includes determining, bya processing device, a model and manufacturer of the computing devicebased on the image of the first surface of the computing device. Themethod further includes rotating the computing device to capture asecond surface of the computing device that is opposite the firstsurface. The method includes capturing an image of the second surface ofthe computing device with the first camera. The method includes movingthe computing device to a second location and capturing an image of athird surface using a second camera and capturing an image of a fourthsurface using a third camera. The method further includes rotating thecomputing device to capture fifth and sixth surfaces that are oppositethe third and fourth surfaces and capturing an image of the fifthsurface using the second camera and capturing an image of the sixthsurface using the third camera. The method further includes selecting,by the processing device, a region of interest in each of the images,the region of interest comprising the computing device having abackground removed. For the plurality of regions of interest asselected, comparing, by the processing device, each region of interestwith a corresponding profile image and identifying defects in eachregion of interest. The method further includes grading, by theprocessing device, a cosmetic appearance of each region of interestbased on the identified defects; and storing the grades of the cosmeticappearance for each region of interest.

A method for inspecting surfaces of an object (“test object”) andgrading the quality of the cosmetic appearance of the object isdisclosed. The method includes locating an object to be inspected (“testobject”) at a specific location. A plurality of images of the testobject are compared with images of a model object. The comparisonincludes a structure-edge detection algorithm to detect scratches andcracks on the surfaces of the test object. The structure-edge detectionalgorithm can be performed utilizing an artificial neural network.Because the lighting is controlled effectively through the domestructure of the system, the structure edge algorithm is more accuratein identifying the defects in the test object rather than reflections onthe surface of the test object. For each image of the test object(corresponding to a surface of the test object), a score is providedbased on how closely the test images match the model images. In thecomparison, the cosmetic grading can analyze discoloration, use opticalcharacter recognition (OCR) to determine whether a mobile device isrefurbished (and if so, by whom), handle stickers (removing or reading asticker, etc.), and determine if there are any differences between twomobile devices of the same model (e.g. logo, sensor size, camera, etc.).The score for each surface is then utilized to form a cosmetic grade forthe test object. The cosmetic grade is output as an indicator of thecosmetic quality of the test object.

FIG. 1 shows a cosmetic inspection system 10 for cosmetic analysis of acomputing device 15, according to an embodiment. In an embodiment, thecomputing device 15 is a secondhand device that is being considered forrefurbishment. In an embodiment, the computing device 15 can be a newlymanufactured device that has yet to be sold.

The system 10 can generally be used to, for example, validate whetherthe computing device 15 is aesthetically acceptable as a candidate forrefurbishment. In an embodiment, a computing device that isaesthetically acceptable may be generally free from cosmetic defectssuch as scratches, dents or chips, cracks, or the like. In anembodiment, the analysis can be part of a refurbishment process.

A computing device 15 tested by the system 10 can include, for example,a smartphone, a tablet, or the like. It is to be appreciated that theseare examples of the computing device and that the computing device canvary beyond the stated list. Examples of other computing devicesinclude, but are not limited to, a smartwatch, a mobile phone other thana smartphone, a personal digital assistant (PDA), a laptop computingdevice, or the like. Furthermore, the maker or manufacturer of thecomputing device 15 is not limited. That is, the system 10 can be usedfor cosmetic analysis of computing devices from different manufacturersso long as a calibration procedure is performed to create a profileimage for the corresponding computing device.

The system 10 includes a display 20 for displaying results of thecosmetic inspection to the user. In an embodiment, the display 20 can bea combined display and input (e.g., a touchscreen). In an embodiment,the display 20 can be a display of a tablet or the like. In such anembodiment, a memory of the tablet can store one or more programs to beexecuted by a processing device of the tablet for inspecting thecosmetic appearance of the assembly computing device 15.

In the illustrated embodiment, the display 20 is secured to housing 25of the system 10. In an embodiment, the display 20 can be separate fromthe housing 25 (i.e., not secured to the housing 25, but positioned nearthe system 10 and electronically connected to the system 10). However,it may be beneficial to secure the display 20 to the housing 25 toreduce a footprint of the system 10.

A platform 30 is utilized to position the computing device 15 within thesystem 10 for validation. The platform 30 enables each computing device15 placed into the system 10 for validation to be placed insubstantially the same location. The platform 30 also moves thecomputing device 15 to different locations to capture images of thecomputing device 15.

FIG. 2 shows a schematic architecture for the system 10 of FIG. 1,according to an embodiment.

The system 10 generally includes a plurality of cameras 100; a movableplatform 105; one or more sensors 110; a processing device 115, memory120, a network input/output (I/O) 125, user I/O 130, storage 135, and aninterconnect 140. The processing device 115, memory 120, networkinput/output (I/O) 125, user I/O 130, storage 135, and interconnect 140can be within the housing 25 in an embodiment. In an embodiment, theprocessing device 115, memory 120, network input/output (I/O) 125, userI/O 130, storage 135, and interconnect 140 can be external from thehousing 25.

The plurality of cameras 100 are arranged in the system 10 to capturedifferent views of the computing device 15. In an embodiment, thecameras 100 are digital cameras. For example, in an embodiment thesystem 10 includes three cameras 100 arranged to capture a top view, andtwo side views (as well as a bottom view and two additional side viewsafter the computing device is rotated by the system 10).

The movable platform 105 can be, for example, configured topneumatically move the computing device 15 in a vertical direction andhorizontal directions. The movable platform 105 also includes ability torotate about a vertical axis and about a horizontal axis to place thecomputing device 15 in different orientations respective of the cameras100. This enables all six surfaces of the computing device 15 to becaptured by the cameras 100.

The one or more sensors 110 can be used to determine when an object isplaced on the movable platform as well as where the movable platform isdisposed relative to the cameras 100 so that the computing device 15 isoriented and located in known locations to provide suitable lighting tocapture high quality images.

The processing device 115 can retrieve and execute programminginstructions stored in the memory 120, the storage 135, or combinationsthereof. The processing device 115 can also store and retrieveapplication data residing in the memory 120. The programminginstructions can perform the method described in accordance with FIGS. 8and 9 below to inspect the cosmetic appearance of the computing device15, and additionally, cause display of one or more graphical userinterfaces (GUIs) on the display 20 showing an outcome of theinspection.

The interconnect 140 is used to transmit programming instructions and/orapplication data between the processing device 115, the user I/O 130,the memory 120, the storage 135, and the network I/O 125. Theinterconnect 140 can, for example, be one or more busses or the like.The processing device 115 can be a single processing device, multipleprocessing devices, or a single processing device having multipleprocessing cores. In an embodiment, the processing device 115 can be asingle-threaded processing device. In an embodiment, the processingdevice 115 can be a multi-threaded processing device.

The memory 120 is generally included to be representative of arandom-access memory such as, but not limited to, Static Random-AccessMemory (SRAM), Dynamic Random-Access Memory (DRAM), or Flash. In anembodiment, the memory 120 can be a volatile memory. In an embodiment,the memory 120 can be a non-volatile memory. In an embodiment, at leasta portion of the memory 120 can be virtual memory.

The storage 135 is generally included to be representative of anon-volatile memory such as, but not limited to, a hard disk drive, asolid-state device, removable memory cards, optical storage, flashmemory devices, network attached storage (NAS), or connections tostorage area network (SAN) devices, or other similar devices that maystore non-volatile data. In an embodiment, the storage 135 is a computerreadable medium. In an embodiment, the storage 135 can include storagethat is external to the user device, such as in a cloud.

FIG. 3 shows a schematic view of a camera system 150 of the cosmeticinspection system of FIG. 1, according to an embodiment.

The camera system 150 includes a plurality of cameras 155, a dome 160, aplurality of top lights 165, a plurality of upper dome lights 170, and aplurality of lower dome lights 175. The combination of components of thecamera system 150 can be used to capture various views of the computingdevice 15 under test. In an embodiment, the images captured using thecamera system 150 can be used by the cosmetic inspection system 10 toanalyze the cosmetic appearance of the computing device 15 and output acosmetic grade to the display 20 of the cosmetic inspection system 10.

The plurality of cameras 155 are representative of, for example, digitalcameras. Although two cameras are shown, it is to be appreciated thatthe camera system 150 includes three cameras, with the third camera 155being disposed in the page (or out of the page), and thus notillustrated. The plurality of cameras 155 can include fixed ISO,f-number parameters, and aperture priority mode. In an embodiment, theplurality of cameras 155 can be configured to capture images that are ator about 2 MB in resolution. Images of this size can, for example, offera significant reduction in the size of the images being stored relativeto prior systems which typically require multiple gigabytes of storageper image.

The dome 160 provides for even light distribution to the computingdevice 15, enabling for generating higher quality images with reducedglare. As a result, the cosmetic inspection system 10 can provide ahigher quality result that is not subject to errors due to glare off thereflective surfaces of the computing device 15. In an embodiment, thedome 160 can be made of a translucent material that allows lighttransmission through the dome 160. For example, in one embodiment thedome 160 is a polyvinylchloride (PVC) material that is white in colorand has a light transmission rate from at or about 0.6% to at or about1.6%. In an embodiment, a light transmission rate of 1.6% can beselected. The dome 160 generally has a smooth outside and a matte insidefinish. A shape of the dome 160 is selected to be generally shaped tomatch a geometry of the computing device 15 being tested by the cosmeticinspection system 10. In an embodiment, once the dome 160 shape isselected, the cosmetic inspection system 10 may be configured to inspecta corresponding computing device type. For example, if the dome 160 isshaped to correspond to a shape of a smartphone, then the cosmeticinspection system 10 may generally be configured for smartphoneinspection, rather than inspection of, for example, laptop devices. Thedome 160 includes a plurality of openings 180 to provide a viewpoint forthe plurality of cameras 155. As a result, the number of openings 180matches the number of cameras 155.

The plurality of lights 165, 170, 175 (i.e., plurality of top lights165, a plurality of upper dome lights 170, and a plurality of lower domelights 175) are disposed at locations outside of the dome 160. Theplurality of lights 165, 170, 175 can be selectively enabled or disabledto provide a specific direction of light (and amount of light) throughthe dome 160 to capture the different views of the computing device 15.In an embodiment, the plurality of lights 165, 170, 175 are lightemitting diode (LED) bar lights. In an embodiment, the plurality oflights 165, 170, 175 can be configured to provide a selected color oflight. In an embodiment, the selected color of light can be white light.

FIG. 4 shows one embodiment of the camera system 150 including threecameras 155 and the dome 160. In the illustrated embodiment, an outlineof the computing device 15 is shown at first and second positions. Atthe first position, the computing device 15 is disposed adjacent to aninner surface of the dome 160 at a vertical location that is relativelynearest to the top camera 155. In both the first and second positions,the computing device 15 is positioned at about a center in a horizontaldirection of the dome 160. The second position is additionallypositioned at about a center in the vertical direction of the dome 160.These positions provide for optimal lighting conditions when capturingimages of the computing device 15, as described further in FIGS. 5-7below.

FIGS. 5-7 show schematic views of the camera system of FIG. 3 configuredto capture different views of the computing device 15, according to anembodiment.

FIG. 5 shows a view when capturing a first top view of the computingdevice 15, according to an embodiment. In the illustrated embodiment,the cosmetic inspection system 10 moves the computing device 15 to aposition in which the computing device 15 is aligned adjacent to aninner surface of a top of the dome 160. The computing device 15 ispositioned horizontally within the dome to be at or about the center ofthe dome 160 in the horizontal direction. To capture the first top view,the plurality of top lights 165 are enabled to emit light toward thecomputing device 15. Although reference is made to the “first top view,”it is to be appreciated that the surface of the computing device 15 thatis captured is dependent upon the orientation of the computing device15. Accordingly, the configuration in FIG. 5 can be used to captureeither the front surface (e.g., the display surface) or the back surfaceof the computing device 15. When capturing the first top view, the imagecan be used to detect cosmetic defects of various materials on the frontor rear surface of the phone, such as glass, metal, or plastic. In anembodiment, when the rear surface of the phone is being captured, theimage as captured can be used to verify a type of the computing device15. For example, computing devices generally include a barcode or otheridentifier that can be captured and analyzed to determine themanufacturer and make of the computing device 15. This can be used todetermine which profile images to use when completing the cosmeticinspection. In an embodiment, the rear surface can be the first surfacecaptured to select the appropriate profile images.

FIG. 6 shows a view when capturing a second top view of the computingdevice 15, according to an embodiment. In the illustrated embodiment,the cosmetic inspection system 10 moves the computing device 15 to aposition in which the computing device 15 is aligned adjacent to aninner surface of a top of the dome 160. The computing device 15 ispositioned horizontally within the dome to be at or about the center ofthe dome 160 in the horizontal direction. In the second top view, theplurality of top lights 165 are disabled and the plurality of upper domelights 170 are enabled. Although reference is made to the “second topview,” it is to be appreciated that the surface of the computing device15 that is captured is dependent upon the orientation of the computingdevice 15. Accordingly, the configuration in FIG. 6 can be used tocapture either the front surface (e.g., the display surface) or the backsurface of the computing device 15. When capturing the second top view,the image can be used to detect cracks in glass materials on the frontor rear surfaces of the computing device 15.

FIG. 7 shows a view when capturing a side view of the computing device15, according to an embodiment. In the illustrated embodiment, thecosmetic inspection system 10 moves the computing device 15 to aposition in which the computing device 15 is positioned horizontallywithin the dome 160 to be at or about the center of the dome 160 in thehorizontal direction and similarly vertically within the dome 160 to beat or about a center of the dome 160 in the vertical direction. In theside view, the plurality of upper dome lights 170 and the plurality oflower dome lights 175 are enabled. Although not shown in the illustratedfigure, the plurality of top lights 165 are disabled in capturing theside view. When capturing the side view, the image can be used to detectdefects in the various materials on the side surfaces of the computingdevice 15, including plastic, matte metal, highly reflective metals, andthe like.

The plurality of views captured (e.g., from FIGS. 5-7) can be used tograde the cosmetic appearance of the computing device 15. Each of theviews contributes to the cosmetic grade. In an embodiment, the cosmeticgrade can be weighted differently depending upon the type of defect. Forexample, in an embodiment, defects on the sides of the computing device15 may be weighted with less significance than defects on the frontsurface (e.g., the display) of the computing device 15. Similarly,defects on the rear surface of the computing device 15 may be weightedwith less significance than defects on the front surface of thecomputing device 15. In an embodiment, the weighting of the defects andthe rules associated with grading the cosmetic appearance may bedetermined by the reseller of the refurbished device. Similarly, if thecosmetic inspection system 10 is used during the initial manufacturingprocess, the weighting can be controlled to provide significanceaccording to rules of the manufacturer of the computing device 15.

FIG. 8 shows a flowchart for a method 200 for inspecting a cosmeticappearance of a test object, according to an embodiment. The method 200can be performed by the cosmetic inspection system 10, and accordingly,by the processing device 115.

At block 205, a test object (e.g., computing device 15 or the like) isloaded into the cosmetic inspection system 10. In an embodiment, thetest object can be loaded by a human operator. In an embodiment, thetest object can be loaded by a robotic or mechanical arm.

At block 210, the movable platform can center the device in a horizontaldirection and vertically move the movable platform to a first position.At the first position, the test object can be flipped so that the a rearsurface of the test object is facing upward (e.g., facing toward a firstcamera of the plurality of cameras).

At block 215, a first top view is captured of the test object. The firsttop view can generally include an identifier (e.g., a barcode or thelike) on the surface of the test object. The first top view captured canbe used to determine a manufacturer and model of the test object. Theinformation as determined can be used to select an appropriate set ofprofile images against which the cosmetic inspection can be compared.

At block 220, the test object can again be flipped so that the frontsurface of the test object is facing upward toward the first camera ofthe plurality of cameras.

At block 225, a second top view of the test object can be captured.

At block 230, the movable platform can be moved and images of the sidesurfaces of the test object captured.

At block 235, the movable platform can return to the starting positionfor unloading of the device.

At block 240, a cosmetic grade is displayed to the user via the displaydevice of the cosmetic inspection system.

FIG. 9 shows a flowchart for a method 250 for grading the cosmeticappearance of a test object, according to an embodiment. The method 250can generally be performed at block 240 of FIG. 8. The method 250 cangrade the cosmetic appearance of the test object using artificialintelligence. The method 250 can use both rule-based and artificialintelligence (AI) solutions. Rule-based solutions include model learningand rule configuration, image capture and processing, and grading. AIsolutions can include artificial neural networks such as, but notlimited to, convolutional neural networks (CNN), data augmentationlearning, incremental learning, and the like. Automatic cosmetic gradinglearns grading criteria using AI or machine learning, which allows foran abbreviated, rule-based training.

At block 255, the method 250 includes determining a region of interest(ROI) of the images being captured. This can be performed for eachcaptured image. Block 255 includes aligning the image of the test objectand the image of the model object. In an embodiment, aligning the imageof the test object with the model object allows for side-by-sidecomparison of the images. In an embodiment, if the images aremisaligned, false positives can be generated when identifying defects.In an embodiment, the image alignment is based on a structure or shapeof the test object instead of color or intensity values. This methodcan, for example, enable comparison of objects that are similar in shapeand size, but different in color. As a result, the grading engine can betrained for a particular size/shape object (e.g., a given profile imageof a computing device), but need not be trained for every possible colorvariation of the object. In an embodiment, the grading engine is alsoless susceptible to false positives resulting from lighting variations.

At block 260, the captured images are compared against the profileimages to identify defects (e.g., scratches, cracks, nicks or dents,discoloration, pin dots, or the like). In an embodiment, this can beperformed using a structure-edge algorithm to identify the edges ofdefects in the test object in the captured images. In an embodiment,aligning the image of the test object with the model object allows forside-by-side comparison of the images. In an embodiment, if the imagesare misaligned, false positives can be generated when identifyingdefects. At block 265, features of the defects can be determined. Thiscan include, for example, identifying a width, length, area, contrast,depth, or the like of the defects as identified at block 260.

At block 270, grading rules can be applied against the features of thedefects determined at block 265. The grading rules can be specific to anentity responsible for the refurbishing of the test object. For example,the reseller of the test object can provide rules that are specific to adevice type. In an embodiment, the grading rules can includeestablishing a relative significance based on the type of the defect,the severity of the defect, the location of the defect, or the like. Thegrading can be accomplished via a grading engine that is trained withgrading criteria and training images (e.g., in a training modeldatabase).

At block 275, a cosmetic grade is produced and can be displayed on thedisplay device of the cosmetic inspection system 10. In an embodiment,the cosmetic grade can be one or more of flawless, lightly-scratched,heavily scratched, or the like. In an embodiment, the cosmetic grade maybe an overall score based on a combination of grading scores for theplurality of sides of an object. In an embodiment, the overall score canbe based on a lowest score of the grading scores for the plurality ofsides of an object. In an embodiment, cosmetic defects can be weightedbased on which side of the object the defect occurs. For example, adefect on the front surface of a test object (e.g., when the test objectis a computing device, the front surface is the display screen of thecomputing device) can be relatively more impactful to the cosmetic gradethan a defect on a side surface of the test object.

FIG. 10 shows a flowchart for a method 300 for grading the cosmeticappearance of a test object, according to an embodiment.

At block 305, the method 300 includes receiving a plurality of images ofa test object. In an embodiment, the plurality of images of a testobject can include an image per surface of the test object. For example,when the test object is a computing device such as a smartphone, theplurality of images can include six images including images of: (1) afront surface (e.g., display), (2) a back surface (e.g., the back of thecomputing device opposite the front surface), (3) side surfacesextending between the front and back surface. In an embodiment, theplurality of images of the test object can include fewer images thannumber of surfaces of the object. In such an embodiment, the images caninclude only major surfaces of the test object. For example, when thetest object is a computing device such as a smartphone, the images caninclude front and back surfaces of the computing device.

In an embodiment, along with the plurality of images received at block305, an indication of a corresponding profile object which can be usedin the comparison can be received at block 305. For example, if the testobject is a computing device such as a smartphone, the method 300 caninclude receiving a model or other identifier that can be used to selectcorresponding profile images for the purposes of identifying defects inthe received plurality of images of the test object.

At block 310, the method 300 includes selecting a region of interest inthe plurality of images. At block 310, the plurality of images can becropped to remove the background so that the test object remains in theimage. Block 310 can also include an alignment of the resulting regionof interest with corresponding profile images. In an embodiment,aligning the image of the test object with the model object allows forside-by-side comparison of the images. In an embodiment, if the imagesare misaligned, false positives can be generated when identifyingdefects. In an embodiment, the image alignment is based on a structureor shape of the test object instead of color or intensity values. Thismethod can, for example, enable comparison of objects that are similarin shape and size, but different in color. As a result, the gradingengine can be trained for a particular size/shape object (e.g., a givenprofile image of a computing device), but need not be trained for everypossible color variation of the object. In an embodiment, the gradingengine is also less susceptible to false positives resulting fromlighting variations.

At block 315, the method 300 includes comparing each region of interestwith a corresponding profile image and identifying defects within eachregion of interest. Because the region of interest was aligned with thecorresponding profile image, a likelihood of receiving a false positiveis reduced. In an embodiment, a structure-edge algorithm can be usedduring the comparison to identify the defects present in the testobject. The structure-edge algorithm can utilize an artificial neuralnetwork, in an embodiment. The structure-edge algorithm generally usesimage brightness variations to identify the edges of defects. Thedefects being identified at block 315 can include, but are not limitedto, scratches, cracks, nicks and dents, discolorations, groups of pindots, and suitable combinations thereof.

At block 320, the method 300 includes grading a cosmetic appearance ofeach region of interest based on the identified defects. There can be aplurality of different grading rules relied upon. The grading rules canbe specific to an entity responsible for the refurbishing of the testobject. For example, the reseller of the test object can provide rulesthat are specific to a device type. In an embodiment, the grading rulescan include establishing a relative significance based on the type ofthe defect, the severity of the defect, the location of the defect, orthe like. The grading can be accomplished via a grading engine that istrained with grading criteria and training images (e.g., in a trainingmodel database). A cosmetic grade can be one or more of flawless,lightly-scratched, heavily scratched, or the like. In an embodiment, thecosmetic grade may be an overall score based on a combination of gradingscores for the plurality of sides of an object. In an embodiment, theoverall score can be based on a lowest score of the grading scores forthe plurality of sides of an object. In an embodiment, cosmetic defectscan be weighted based on which side of the object the defect occurs. Forexample, a defect on the front surface of a test object (e.g., when thetest object is a computing device, the front surface is the displayscreen of the computing device) can be relatively more impactful to thecosmetic grade than a defect on a side surface of the test object.

At block 325, the method 300 includes storing the grades of the cosmeticappearance for each region of interest.

Examples of computer-readable storage media include, but are not limitedto, any tangible medium capable of storing a computer program for use bya programmable processing device to perform functions described hereinby operating on input data and generating an output. A computer programis a set of instructions that can be used, directly or indirectly, in acomputer system to perform a certain function or determine a certainresult. Examples of computer-readable storage media include, but are notlimited to, a floppy disk; a hard disk; a random access memory (RAM); aread-only memory (ROM); a semiconductor memory device such as, but notlimited to, an erasable programmable read-only memory (EPROM), anelectrically erasable programmable read-only memory (EEPROM), Flashmemory, or the like; a portable compact disk read-only memory (CD-ROM);an optical storage device; a magnetic storage device; other similardevice; or suitable combinations of the foregoing.

In some embodiments, hardwired circuitry may be used in combination withsoftware instructions. Thus, the description is not limited to anyspecific combination of hardware circuitry and software instructions,nor to any source for the instructions executed by the data processingsystem.

The terminology used herein is intended to describe embodiments and isnot intended to be limiting. The terms “a,” “an,” and “the” include theplural forms as well, unless clearly indicated otherwise. The terms“comprises” and/or “comprising,” when used in this Specification,specify the presence of the stated features, integers, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, operations, elements,and/or components.

It is to be understood that changes may be made in detail, especially inmatters of the construction materials employed and the shape, size, andarrangement of parts without departing from the scope of the presentdisclosure. This Specification and the embodiments described areexamples, with the true scope and spirit of the disclosure beingindicated by the claims that follow.

What is claimed is:
 1. A method, comprising: receiving, by a processingdevice, a plurality of images of a test object, the plurality of imagesincluding a plurality of surfaces of the test object; selecting, by theprocessing device, a region of interest in each of the plurality ofimages, the region of interest comprising the test object having abackground removed; for the plurality of regions of interest asselected, comparing, by the processing device, each region of interestwith a corresponding profile image and identifying defects in eachregion of interest; grading, by the processing device, a cosmeticappearance of each region of interest based on the identified defects;and storing the grades of the cosmetic appearance for each region ofinterest.
 2. The method of claim 1, wherein the identifying defects ineach region of interest comprises a structure-edge algorithm to identifystructural differences between the plurality of images of the testobject and the corresponding profile images.
 3. The method of claim 1,wherein the defects include scratches, cracks, nicks and dents,discoloration, pin dots, and combinations thereof.
 4. The method ofclaim 1, comprising determining one or more features of the defects ofeach region of interest as identified, wherein the one or more featurescomprise dimensions of a defect, area of a defect, contrast of a defect,depth of a defect, and combinations thereof.
 5. The method of claim 4,wherein grading the cosmetic appearance of each region of interest isbased on a severity of the identified defects as determined from the oneor more features of the identified defects.
 6. The method of claim 1,comprising outputting, by the processing device, the grades of thecosmetic appearance for each region of interest on a display.
 7. Themethod of claim 1, wherein grading the cosmetic appearance of eachregion of interest based on the identified defects comprises weighting asignificance of a defect based on its location on the plurality ofsurfaces of the test object.
 8. The method of claim 1, wherein gradingthe cosmetic appearance of each region of interest based on theidentified defects comprises weighting a significance of a defect basedon a defect type, wherein the types of defects include scratches,cracks, nicks and dents, discoloration, and pin dots.
 9. The method ofclaim 1, comprising: aligning the plurality of images with correspondingprofile images.
 10. A method, comprising: capturing an image of a firstsurface of a computing device with a first camera; determining, by aprocessing device, a model and manufacturer of the computing devicebased on the image of the first surface of the computing device;rotating the computing device to capture a second surface of thecomputing device that is opposite the first surface; capturing an imageof the second surface of the computing device with the first camera;moving the computing device to a second location and capturing an imageof a third surface using a second camera and capturing an image of afourth surface using a third camera; rotating the computing device tocapture fifth and sixth surfaces that are opposite the third and fourthsurfaces; capturing an image of the fifth surface using the secondcamera and capturing an image of the sixth surface using the thirdcamera; selecting, by the processing device, a region of interest ineach of the images, the region of interest comprising the computingdevice having a background removed; for the plurality of regions ofinterest as selected, comparing, by the processing device, each regionof interest with a corresponding profile image and identifying defectsin each region of interest; grading, by the processing device, acosmetic appearance of each region of interest based on the identifieddefects; and storing the grades of the cosmetic appearance for eachregion of interest.
 11. The method of claim 10, wherein the firstsurface is a rear surface of the computing device and the second surfaceis a front surface of the computing device, the second surface being adisplay of the computing device.
 12. The method of claim 10, wherein thethird, fourth, fifth, and sixth surfaces are side surfaces of thecomputing device.
 13. The method of claim 10, wherein the defectsinclude scratches, cracks, nicks and dents, discoloration, pin dots, andcombinations thereof.
 14. The method of claim 10, comprising determiningone or more features of the defects of each region of interest asidentified, wherein the one or more features comprise dimensions of adefect, area of a defect, contrast of a defect, depth of a defect, andcombinations thereof.
 15. The method of claim 14, wherein grading thecosmetic appearance of each region of interest is based on a severity ofthe identified defects as determined from the one or more features ofthe identified defects.
 16. The method of claim 10, comprisingoutputting, by the processing device, the grades of the cosmeticappearance for each region of interest on a display.
 17. The method ofclaim 10, wherein grading the cosmetic appearance of each region ofinterest based on the identified defects comprises weighting asignificance of a defect based on its location on the plurality ofsurfaces of the test object.
 18. The method of claim 10, wherein gradingthe cosmetic appearance of each region of interest based on theidentified defects comprises weighting a significance of a defect basedon a defect type, wherein the types of defects include scratches,cracks, nicks and dents, discoloration, and pin dots.
 19. The method ofclaim 10, wherein the identifying defects in each region of interestcomprises a structure-edge algorithm to identify structural differencesbetween the plurality of images of the test object and the correspondingprofile images.
 20. The method of claim 10, comprising: aligning theplurality of images with corresponding profile images.